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SIR: 



APPEAL BRIEF PURSUANT TO 37 CFR 1,191 



Real party in interest 
The real parties in interest are AT&T Corp. the assignee of this application, and 
the inventors identified above. 



Claims 1-14 stand rejected. 

• Claims 1-6, and 8-13 are rejected under 35 USC 103 as being unpatentable over 
Peters, US Patent 5,715,334, in view of Burfeind et al, US Patent 6.052,648. 

• Claims 7 and 14 are rejected under 35 UCS 103 as being unpatentable over the Peters 
reference in view of the Burfeind et al reference and further in view of allegedly 
admitted prior art. 



Related appeals and interferences 
There are no related appeals or interferences. 



Status of claims 



08/06/2004 HHEKOHEN 00000014 09721884 



01 FCsl402 



330.00 OP 



Status of amendments 

An amendment after final was filed together with the Notice of Appeal on June 9, 
2004 (received in the USPTO on 6/1 1/2004). No advisory action was received to 
indicate the status of the amendment, but a conversation with the Examiner revealed that 
(a) for reasons unknown to the Examiner, the filed amendment is not in the case, and (b) 
because an amendment to claim 1 was included (albeit a trivial one) the Examiner would 
not have entered the amendment. 

Summary of invention 

This invention is directed to a new visualization platform for the interactive 
exploration of large datasets that arrive at real time. The processing module are designed 
to operate in a streaming mode, where data is received, accepted, processed and displayed 
(stages of handling data), and all stages are concurrently and continuously handling data. 
The invention integrates a collection of relevant visualization techniques to provide a new 
visual metaphor for viewing large datasets, where the displayed objected (that represent 
results of analysis of the incoming data) are continually updated. The system is capable 
of providing comprehensive support for data exploration that integrates the large-scale 
data visualization with querying, browsing, and statistical evaluation. 

Issues 

The primary issue whether the rejected claims are obvious in view of Peters, US 
Patent 5,715,334, in view of Burfeind et al, US Patent 6.052,648. This corresponds to the 
issue of whether the Examiner is correct in his reading of what the Peters and Burfeind 
references teach, and the application of these teachings to the claims at hand. 

Grouping of claims 

A) Claims 1, 2, 3, 7, 8, 9, 10, and 14 

B) Claim 4, 11 

C) Claim5, 12 

D) Claim 6, 13 
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Argument 

Group A: Claims 1. 2, 3, 7, 8, 9, 10, and 14 
The Peters reference: 

The Peters reference teaches an image information enhancement technique 

through "novel 'smoothing' and 'detail enhancement'" techniques for processing digital 

images." Col. 3, lines 14-16. The described method 

makes possible a separation of the image information into three 
objective distinct non-random intensity groups of 'large image feature' 
and small 'special detail' and 'intensity detail', and one group of 
random intensity information of the 'image noise'." Col. 3, lines 23- 
27. 

The precise method by which the technique achieves its results is not pertinent to 

this appeal. Suffice it to say that the presented image is enhanced by operation of some 

processing modules that perform both smoothing and detail enhancement. Additionally, 

the reference teaches a new detail filter that 

produces images which characterize the intensity distributions with an 
image. It provides a tool for a novel method of image information 
analysis and classification based on the concept that any image 
communicates information only through image contrasts which are 
intensity variations between certain pixel arrays with in the total data 
matrix. Col. 6, lines 59-65. 

In trying to classify the information that an image contains, Peters concludes that 

images in general can be divided into a non-random structural class of features and detail, 

and a random class of noise. Col. 14, line 64 - col. 15, line 2. Peters further concludes 

that a 

feature is an image component accessible to visual pattern recognition, 
i.e., it is larger than ~ 10% of the picture widths, and has a contrast 
range of more than -5% of the visual intensity range, and each must 
fall within no more than 15-20 intensity levels in order to be recognized 
as a pattern. Col. 15, lines 2-8. 

All other structural image components are summarized by Peters as "details," but he 

nevertheless subdivides this category as follows: 

small high contrast components (<10% PW, >5% BW) are 
characterized as spatial (sic) details, col. 15, lines 8-10 

whereas 
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low contrast components of any size (<5% BW) are characterized as 
intensity details. Col. 15, lines 10-11. 

In addition, Peters teaches, 

digital images contain noise of various origins. Only the high frequency 
noise components at the level of a few pixels are visually perceivable 
as random intensity fluctuations. Other low frequency noise 
components, which have the character of intensity details, may be 
recognizable only after eliminating (averaging, smoothing) the high 
frequency components. Col. 15, lines 12-18. 

The above is visually illustrated in FIG. 5, showing features, special details, intensity 

details, and noise. The upper left corner of this two dimensional depiction corresponds to 

maximum intensity and spatial dimension, and the lower right corner corresponds to 

minimum intensity and spatial dimension. 

FIG. 6 is a tutorial that has three parts. 

The top part of FIG. 6, titled "Image Information Analysis" depicts a process or, 
more accurately, images at different stages of a process. At the left edge, a hypothetical 
"original" image is shown, which is the incoming data of this illustrative tutorial. A 
horizontal dashed line across the image (slightly below the midpoint of the image) is 
extended to what appears to be a window that shows (not unlike what an oscilloscope 
might show) the signal that corresponds to the image pixels along this dashed line. It 
shows a signal that has a large value for about the first half the time - corresponding to a 
primarily white portion of the image, followed by a low value for the remainder of the 
time - corresponding to the primarily dark portion of the image. 

Slightly below and to the right of this window there is a second "oscilloscope 
window" that shows both the hysteresis range and the smoothed image. To the right of 
the second window there is a third "oscilloscope window" that shows the detail image, 
and slightly to the right and above the third window there is a fourth "oscilloscope 
window" that shown the signal that corresponds to the enhanced version of the signal 
originally obtained from the hypothetical image at the left of the FIG. Finally, to the 
right of the fourth window there is an image that is an enhanced version of the image 
presented at the extreme left. 

The middle portion of FIG. 6, titled "Image Information Classes" depicts the four 
digital information classes that Peters decided to employ. The four images that are 
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presented show what the hypothetical image (presented at the left of the FIG's top 
portion) would look like if only the specified information were depicted. Thus, the left- 
most image depicts only the noise components of the original image, the image to the 
right of the noise image depicts only spatial detail contained in the original image. To 
the right of the spatial detail image there is an image of the intensity detail components 
of the original image. Lastly, to the right of the intensity detail image, at the right most 
part of the middle portion of FIG. 6, there is an image of the features contained in the 
original image. 

The bottom part of FIG. 6, titled "Quantitation of Information and Quality," is a 
pictorial representation of the data classes that are found in the image. Termed an 
"information cube," it shows pictorially (a two-dimensional image of a three-dimensional 
cube that circumscribes three-dimensional structures) the amount of data that belongs to 
classes "F" (features), "I" (intensity detail), "S" (spatial detail), and "N" (noise), in order 
when viewed from the bottom of the cube toward its top. 

Discussion of assertions based on Peters - claim 1: 

In rejection claim 1, the Examiner asserted that Peters teaches: 

• a "visualization interface," which is a module, or a technique used in a module, is 
taught by Peters teaches in FIG. 5 and the associated text at col. 14, line 63 to col. 
14, line 36, 

• "a plurality of processing tools" is taught in FIG. 6, and the associated text at col. 
18, line 59 to col. 19, line 1 

• "means that enables streaming the data to and through one or more said 
processing tools to create data results for updating one or more objects, which one 
or more object may be displayed by the visualization interface" is taught at FIG. 
23, and the associated text at col. 29, lines 22-65. 

Applicants disagree, respectfully submitting that the Examiner is misreading the 
teachings in the cited FIGs. and passages. 

As for the first assertion, as indicated above, FIG. 5 is a two dimensional 
depiction of the different classes of objects that Peters concluded are useful in classifying 
images. FIG. 5 neither describes nor suggests any tool, or processing module. It also 
does not describe or suggest a "visualization interface." 
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The Peters reference might have a visualization interface that is used to display an 
enhanced version of an applied image, and/or used to display the "information cube." 
However, one cannot infer it from FIG. 5 and the associated text cited by the Examiner. 
In other words, strictly speaking, the Examiner erred. 

The same applies to the second assertion, where the Examiner's refers to FIG. 6, 
and the text at col. 8, line 59 to col. 19, lines 1. Applicants note that the quoted passage 
refers to a tool for creating the information cube, but that is not a plurality of tools . The 
reference to FIG. 23 refers to the same tool, 

A significantly stronger case for patentability exists in connection with the third 
and fourth clauses of claim 1 . 

The fourth clause of claim 1 specifies 

means that enables streaming the data to and through one or more of 
said processing tools to create data results for updating one or more 
objects, which one or more objects may be displayed by the 
visualization interface, (emphasis supplied) 

The highlight phrases specify central attributes of the defined means. However, in 

applicants' view, the Examiner failed to give proper consideration to the highlighted 

phrases and, consequently, erroneously asserted that Peters teaches such a means. 

Almost without exception software tools operate on data that is stored in a 
specified memory location. When one tool operates on the data, no other tool is 
permitted access to that data. Sometimes, a system calls for a tool to operate on data, 
modify it, and make subsequent passes, modifying the data with each pass. A single pass 
through a data set and even multiple passes through a data set are not a streaming 
operation. In contradistinction, a streaming operation is one where as soon as an element 
of data is available as a result of operation by one tool, it is applied to another tool. 

The first software embodiment of streaming known to applicants is the "pipe" 
command in the UNIX operating system (specified by the "|" character). This command 
is an example of a "means that enables streaming." For example, the UNIX command 
line 

who | wc 

streams the result of the "who" tool to the "wc" tool. 
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Recently, in the context of the Internet, the concept of streaming was broadened 
somewhat to include chunks of data rather that just an element of data (such as when 
downloading video, before the download of the entire video is completed, it begins to be 
displayed), and those chunks are temporarily stored. 

The tools, or tools, of Peters are clearly not used in the streaming mode, and there 
is no "means that enables streaming." Indeed, since the notion of having separate passes 
through the data is mentioned a number of times in the Peters reference (i.e., col. 13, lines 
26, 50, 57; col. 17, lines 60-65), effectively proves that no streaming takes place in the 
Peters reference. Consequently, Peters has need for a "means that enables streaming." 

The Examiner points to FIG. 23, and the associated text at col. 29, lines 22-65. 

Applicants respectfully submit, however, that neither FIG. 23 nor the cited text describe 

or suggest streaming, and certainly not a "means that enables streaming." What FIG. 23 

does depict is a plurality of "information cubes," and the cited text states: 

The definition in digital images of any origin and content through PAIP 
of defined contrast classes and the quantitation of these classes 
provides a unique and new tool for image quality quantitation. The four 
intensity information classes can be schematically represented in an 
information cube (FIG. 23) which depicts the relative significant 
intensity (z coordinate) over the image (x and y coordinates). 

Respectfully, that does not demonstrate that any streaming takes place in Peters, or that a 

"means that enables streaming" exists, or is suggested. The cited text neither suggests 

streaming nor precludes. However, the passage at col. 29, lines 35-37 clearly 

characterizes what FIG. 23 depicts, by saying 

In FIG. 23 info-cubes are presented of some of the microscopy data sets 
depicted in FIGS. 8-20. (emphasis supplied) 

Clearly, therefore, FIG. 23 shows "information cubes" of various images that result from 

different data . In other words, each cube is an example of a specific input image. 

Consequently, it is clear that the FIG. 23 images represent different data and do not 

represent, or suggest, streaming. 

Actually, even if the FIG. 23 images were depicting different stages of a 

processing chain responsive to a single set of data, it would still not prove the existence 

of streaming, since it can be simply a representation of the different stages that result (at 

different times) from different passes through the entirety of the data set. 
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In addition, the fourth clause of claim 1 specifies that the data is created by 
passing data through a plurality of tools, in a streaming manner, with the aid of a "means 
for enabling streaming" to update one or more objects adapted for display by the 
visualization interface. That is, the means for enabling streaming, and the tools, work to 
update objects that have been created previously, for example, from previous data, 
whether by streaming or otherwise. This in contrast to tools that create, or recreate 
objects ab initio. The Peters reference teaches, or suggests nothing relating to updating 
(existing) objects to be displayed. Actually, the clear thrust of the Peters teachings is that 
data which represents an image is provided, the data is processed, the processed data can 
be displayed as an enhanced image, and an "information cube" is created that can also be 
displayed. In short, no updating of existing objects. 

Moreover, and linked to the fourth clause of claim 1 , it should be noted that the 
notion of streaming is always associated with real-time systems. That is, real-time 
systems are systems that (1) receive data on a continuous basis, (2) immediately handle 
the data, and (3) dispose of it. Claim 1 indeed specifies that the defined system is a real- 
time system, and this specification in the preamble is a limitation of the claim by virtue of 
the definition of the means that enables streaming. In contra distinction, there is no 
indication anywhere in the reference that the Peters system is a real-time system, and the 
Examiner has not pointed to any. 

It bears reiterating that in the Peters environment, there is no need for streaming 
of data through tools (which, as indicated above, Peters neither describes nor suggests) 
and there is no need for updating of rendered objects (which Peters neither describes nor 
suggests) because there in additional data that is incoming that might affect the rendered 
object. It is not surprising therefore that Peters neither describes nor suggests such 
features. Moreover, there is no motivation for incorporating such features into the Peters 
arrangement. 

For these reasons, it is clear that the Examiner erred in his asserting a 
correspondence between the teachings of Peters and the fourth clause of claim 1 . 
The Burfeind reference : 

The Examiner admits that Peters has nothing to correspond to the third clause of 
claim 1, but asserts that Burfeind does. Specifically, the Examiner asserts that in FIGS. 5 
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and 6, and in the associated text at col. 5, lines 2-13, Burfeind teaches a "means for 
accessing a plurality of data files that had been converted to a uniform self-describing 
format." Applicants respectfully disagree and, frankly, applicants are at a loss. 

The Burfeind FIG. 5 "is an. exemplary representation of a virtual set according to 
one embodiment of the invention." Col. 2, lines 5-6. In col. 3, lines 27-29, the reference 
is more specific; saying that "[A]s shown in FIG. 5, weather forecaster 34 is the subject, 
and is shown in room or portion thereof 36." 

The Burfeind FIG. 6 shows just the outline of the room and the outline of the 
weather forecaster. 

It is hard to understand how the Examiner points to FIG 5 and 6, which depict 
different images, and asserts that they are "means." While one must surmise that these 
images come from stored files, it remains hard to understand how such images describe 
ANY format, even if the images are stored in some files. Compounding the puzzle, it is 
hard to understand how such images describe the format of files, how such images 
describe or suggest files that have a self-describing format , or how such images describe 
or suggest files that have a uniform self-describing format. 

The text cited by the Examiner refers to FIG. 9, and not to FIGS. 5 and 6, but a 
perusal of the text that does related to FIGS. 5 and 6 neither describes nor suggest any 
format for any files. 

As for FIG. 9, it depicts a system. The aforementioned cited text states: 

Referring back to FIG. 9, after a virtual set has been generated by video 
camera 44, and a chroma-key mask has been generated by chroma-key 
device 58, both of these signals are fed into the real-time texture map 
device 60. The mask output of device 58 is input as the alpha input 
(shown in FIG. 6), and the video feed output of camera 44 is input as 
the video input (shown in FIG. 5). Texture map device 60 converts 
those parts of the video feed that correspond to the on color of the alpha 
input to a texture memory, which is a digital representation of the 
weather forecaster stored in the memory of the computer in a format 
having the ability to be manipulated by the graphics processors of the 
computer. 

To the Examiner's credit, the one and only occurrence in the entire Burfeind patent of the 
word "format" is found in the above quoted patent, which teaches that a digital 
representation of the weather forecaster is stored in memory of the computer in a format 
having the ability to be manipulated by the graphics processor of the computer. This 
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teaching, however, does not support the Examiner's assertion, because it teaches 
essentially nothing about the format of the file that stores the digital representation of the 
weather forecaster. Aside from the essentially non-existent limitation that,the format of 
the file must be such that another program can manipulate the file, anything goes. As 
long as the graphics processors of the computer can manipulate the information, the 
requirements of the Burfeind teachings are satisfied. This is valid whether a file is of a 
self-describing format, or not. 

At best, one might say that the Burfeind reference, as a whole, depict a system 
that employs files to create a composite image. The associated text cited by the Examiner 
supports such a characterization but, as indicated in the response to the previous Office 
Action, and as explained above, the notion of employing files to create a composite 
image neither teaches nor suggests that the files are ones that have a "self describing 
format." It is noted that, moreover, the notion employing files to create a composite 
image does not teach or suggest a "uniform self-describing format" (emphasis supplied), 
and it also does not teach or suggest that such files were converted to such a format from 
some other format. 

Based on the above, it is respectfully submitted that the Examiner erred in 
asserting that Burfeind teaches a "means for accessing a plurality of data files that had 
been converted to a uniform self-describing format." 

Additionally, there is nothing in the Peters reference that suggests the notion of 
self-describing format of files. Such a format will not improve the operation of the Peters 
arrangement at all, because the data that comes is of a known variety, and the tools are 
adapted to work with that data. There is no need for identifying the format of the data. 
Hence, there is absolutely no motivation for modifying the Peters arrangement in thhe 
manner suggested by the Examiner (even if the Burfeind reference were teaching what 
the Examiner asserts). 

In short, it is respectfully submitted that a careful reading of the Peters and 
Burfeind references reveals that the claim 1 limitations define subject matter that is not 
obvious in view of the Peters and Burfeind combination of references. The Examiner 
erred in not taking proper cognizance of the limitations found in claim 1 , which make 
claim 1 clearly patentable. 



10 



Discussion — claim 8: 

Independent claim 8 is a method claim that includes the above-discussed 
limitations of 

(1) being a real-time method; 

(2) data files have a self-describing format; 

(3) data files have a uniform self-describing format; 

(4) streaming of data; and 

(4) streaming of data to create results for updating one or more objects. 

In light of the above limitations, for the reasons expressed above in connection 
with claim 1 ? it is respectfully submitted that the Examiner erred in not taking proper 
cognizance of the limitations found in claim 8, which make claim 8 clearly patentable. 
Group B: Claims 4, and 1 1 

Claim 4 specifies that the visualization interface "can access the data results as 
the processing tools are working on the data." This is different from prior art 
systems, where a tool that operates on the data of a file does not make any data 
available until the module finishes the processing. 

In rejecting the claims, the Examiner cited FIG. 6 and the text at col. 18. line 
59 to col. 19, line 1. Applicants respectfully disagree that either FIG. 6 or the cited 
text suggests this capability. The substance of FIG. 6 was discussed in detail above, and 
in the interest of brevity it is not repeated here. With reference to the limitation discussed 
herein, it may be summarized that FIG. 6 shows image information analysis results, but 
there is nothing in the FIG. or in any associated text pertaining to any timing relative to 
processing, or relative to when processing results data is made available. 

As for the specifically cited passage, it states: 

The data reduction to individual intensity information classes provides 
a new and unique tool for the analysis of image information and 
quantitation of the image information content and image quality. The 
application of this tool in digital image processing lead to the discovery 
that all known contrast mechanisms of any technical imaging 
equipment (microscopes, telescopes, photographic and video cameras, 
medical imaging technologies, etc.) establish one of the three basic 
intensity variations found as structural classes in digital images 

Clearly, the above-quoted passage does not speak of any timing relative to processing, or 

relative to when processing results data is made available. 
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It is respectfully submitted that the references cited by the Examiner do not 
describe or suggest a "visualization interface can access the data results as the 
processing tools are working on the data," and the Examiner erred in asserting that the 
Peters reference teaches the limitations of claims 4 and 1 1 . 
Group C: Claims 5, and 12 

Claims 5 specifies that the visualization interface enables selection of a portion 
of the data results such that data corresponding to the selected portion may be 
accessed and processed, in real-time, to create second data results that are displayed 
on the visualization interface. 

In rejecting claim 5, the Examiner asserted that the passage in the col. 6, line 59 to 

col. 7, line 6, of the Peters reference teaches this limitation. Applicants respectfully 

disagree. The cited passage states: 

The new detail filter produces detail images which characterize the 
intensity distributions within an image. It provides a tool for a novel 
method of image information analysis and classification based on the 
concept that any image communicates information only through image 
contrasts which are intensity variations between certain pixel arrays 
within the total data matrix. An area of certain contrast is defined by 
the differences between its average intensity and the surrounding 
intensity irrespectively of the overall intensity variations (background). 

The passage teaches that an image has intensity variations, which produce 
contrast, but it neither teaches nor suggests the notion of (1) selecting a portion of the 
data results, (2) accessed in real time, (3) creating a second data result therefrom, and (4) 
displaying the second data results separately. 

In short, it is respectfully submitted that the Examiner erred in making an 
assertion that is not supported by the passage cited by the Examiner and, indeed is not 
supported by any other passage in the cited references. 
Group D: Claims 6, and 13 

Claim 6 specifies that the processing tools enable creation of new processing 
expressions that are compiled and dynamically linked to the processing tools. 
Applicants believe that this is a quite unexpected and significant innovation. 

In rejecting the claim, the Examiner asserted the Peters teaches this limitation in 
FIG. 6 and in the associated text at col. 1 8, line 59 to col. 19, line 5. Applicants 
respectfully disagree. 
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As to the content of FIG. 6, the reader's attention is respectfully to the discussion 
above. In applicants' view, FIG. 6 clearly does NOT describe or suggests creating 
processing expressions, and certainly not creating new processing expressions. 
Furthermore, FIG. 6 neither describes no suggests new processing expressions that are 
compiled and dynamically linked to the processing tools. As for the text cited by the 
Examiner, it addresses the fact that the Peters' invention discloses individual intensity 
information classes and that this approach to analysis of images lead "to the discovery 
that all known contrast mechanisms of any technical imaging equipment ... establishes 
one of the three basic intensity variations found as structural class in digital images." 
None of this (the FIG., the text, and/or their combination) teaches, or suggests, the notion 
of creating new processing expressing, or even allowing the creation of new processing 
expressions. Therefore, it is respectfully submitted that claim 6 is not obvious in view of 
the cited references, taken singly or in combination, and that the Examiner erred in his 
assertions. 

In light of the above arguments, it is respectfully requested that the Board return 
the case to the Examiner with directive to rescind the rejections and to allow all claims. 



Respectfully, 
Emden Gansner 
Eleftherios Koutsofios 
Stephen Charles North 
Russell N. Truscott 





Henry T£ Brendzel 
Reg. Wo. 26,844 
Phone (973) 467-2025 
Fax (973)467-6589 
email brendzel@comcast. net 



13 



APPENDIX 
(CLAIMS INVOLVED IN THE APPEAL) 

1. A real-time large-scale visualization system comprising: 
a visualization interface; 

a plurality of processing tools; 

means for accessing a plurality of data files that had been converted to a 
uniform self-describing format; and 

means that enables streaming the data to and through one or more of said 
processing tools to create data results for updating one or more objects, which one or 
more objects may be displayed by the visualization interface. 

2. The invention of claim 1 wherein the visualization interface provides linked 
views of the data results. 

3. The invention of claim 2 wherein the visualization interface is capable of 
presenting a statistical two-dimensional view, a pixel-oriented two-dimensional view, and 
a dynamic three-dimensional detailed view. 

4. The invention of claim 1 wherein the visualization interface can access the 
data results as the processing tools are working on the data. 

5. The invention of claim 1 wherein the visualization interface enables 
selection of a portion of the data results such that data corresponding to the portion 
selected may be accessed and processed in real-time to create second data results that 
are displayed on the visualization interface. 

6. The invention of claim 1 wherein the processing tools enables creation of 
new processing expressions that are compiled and dynamically linked to the 
processing tools. 

7. The invention of claim 1 wherein the data is accessed using Direct 10. 
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8. A method of visualizing large-scale data in real-time comprising: 
accessing a plurality of data files that had been converted to a uniform self- 
describing format; 

streaming the data to and through one or more processing tools to create data 
results for updating one or more objects, which one or more objects are adapted for 
display; 

displaying said one or more objects on a visualization interface. 

9. The invention of claim 1 wherein the visualization interface provides linked 
views of the data results. 

10. The invention of claim 2 wherein the visualization interface is capable of 
presenting a statistical two-dimensional view, a pixel-oriented two-dimensional view, 
and a dynamic three-dimensional detailed view. 

11. The invention of claim 1 wherein the visualization interface can access the 
data results as the processing tools are working on the data. 

12. The invention of claim 1 wherein the visualization interface enables 
selection of a portion of the data results such that data corresponding to the portion 
selected may be accessed and processed in real-time to create second data results that 
are displayed on the visualization interface. 

13. The invention of claim 1 wherein the processing tools enables creation of 
new processing expressions that are compiled and dynamically linked to the 
processing tools. 

14. The invention of claim 1 wherein the data is accessed using Direct 10. 
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